In recent years, there has been a dramatic upsurge in the popularity of electronic communication in business and home applications. The number of networks and the volume of data continue to increase at a rapid rate. To cope with the ever-increasing demand for faster, more secure and more far-reaching networks, a variety of network appliances are being used to meet these demands.
Network appliances in the network are constantly monitored to identify problems and upgrade opportunities in order to help ensure the integrity and to maintain the optimal performance of a network. Statistical data are often used to accomplish these goals. For example, a network administrator may monitor the throughputs of a message protector appliance in different time ranges to determine whether the appliance is performing properly and whether the appliance is being overloaded. Statistical data of parameters such as the number of emails, the total size of the processed emails, and the number of emails containing virus or exploits are useful to an administrator for analyzing the performance of the message protector appliance.
One common method of gathering statistical data on network appliances is by logging data. A logging system is typically used to record data into log files. The data is used to record the details of all transactions associated with the network appliances. The details of a transaction are generally included in an entry, which is usually identified by a time. Later, if statistical data of certain time period are requested, the system parses the log files and calculates the parameters of interest by analyzing each entry that falls in the time period.
A significant drawback of this approach is that huge amount of data are involved. For example, a message protector appliance can process tens of thousands of emails per hour. To determine how many emails were processed in any one-hour period, the system has to parse through tens of thousands of log records, which is a resource-intensive and time-consuming proposition. Statistical data for longer periods such as months or years would take even longer to compute. Thus, in such a system, providing real-time statistical data is simply impractical.
Another method for gathering statistical data on network appliances is by entering statistically significant data of transactions into an actual database. There are at least two major drawbacks to this method. First, having an actual database adds significant monetary cost and operational resource requirements to a network for merely monitoring network appliances. Second, performing real-time database operations requires a level of computational overhead that is all but prohibitive in most applications.
An extendable monitoring system that can efficiently and reliably provide statistical data on network appliances eludes those skilled in the art.